Fix CLI token source --profile fallback with version detection by mihaimitrea-db · Pull Request #1605 · databricks/databricks-sdk-go

mihaimitrea-db · 2026-03-30T13:33:44Z

Summary

Fix the broken --profile fallback in CliTokenSource by replacing error-based detection with version-based CLI detection at init time.

Why

The --profile flag on databricks auth token is a global Cobra flag (defined as a persistent flag on the root command). Old CLIs (< v0.207.1) silently accept it — they don't report "unknown flag: --profile" but instead fail later with "cannot fetch credentials". This means the existing isUnknownFlagError check (config/cli_token_source.go:120) never matches, and the --host fallback is dead code.

This was verified by testing against CLI v0.207.0 vs v0.207.1:

v0.207.0: databricks auth token --profile workspace → Error: init: cannot fetch credentials (not "unknown flag")
v0.207.1: databricks auth token --profile workspace → returns a valid token

Approaches considered

Three approaches were evaluated for detecting whether the installed CLI supports --profile:

Error-based detection (try-and-retry) — the current approach on main. Run databricks auth token --profile <name> and check whether the error contains "unknown flag: --profile". This is broken: because --profile is a global Cobra flag, old CLIs accept it silently and fail with a different error ("cannot fetch credentials"), so the fallback to --host never triggers.
--help flag parsing (databricks auth token --help + substring matching) was rejected because the --help output format is not a stable API. More importantly, --profile would appear in --help output even on old CLIs that don't actually implement profile-based token lookup — it shows up because it's a global persistent flag, not because the auth token subcommand uses it. This approach has the same fundamental flaw as error-based detection.
Version detection (databricks version + semver comparison) — the approach taken here. Run databricks version at init time, parse the semver (e.g., "Databricks CLI v0.207.1"), and compare against known minimum versions for each flag. This is reliable because the version string is a stable output format, and the mapping between flags and CLI versions is well-defined (databricks/cli#855 for --profile in v0.207.1). If version detection fails, the SDK falls back to the most conservative command (--host only).

References

--profile support added in CLI v0.207.1: databricks/cli#855 (Oct 2023)

What changed

Interface changes

None. CliTokenSource is not part of the public API surface.

NewCliTokenSource now takes context.Context as its first parameter, needed for exec.CommandContext when running databricks version. This is consistent with every CredentialsStrategy.Configure method in the codebase, and the single caller (auth_u2m.go) already has ctx in scope.

Behavioral changes

When cfg.Profile is set but the CLI is too old (< v0.207.1), the SDK now correctly falls back to --host. Previously this fallback was dead code.
A warning is logged when the --profile flag is not supported and the SDK falls back to --host.

Internal changes

cliVersion type: semver parsing with AtLeast() comparison and String() formatting.
getCliVersion(ctx, cliPath): runs databricks version and parses the output.
parseCliVersion: parses "Databricks CLI v0.207.1" → cliVersion{0, 207, 1}.
resolveCliCommand: bridges version detection and command building. Falls back to zero version on detection failure.
buildCliCommand: pure function — takes a version, returns a single resolved command. No exec calls, easy to test.
CliTokenSource simplified to a single cmd []string field. No hostCmd, no runtime fallback.
Token() is now one line: return c.execCliCommand(ctx, c.cmd).
Removed: isUnknownFlagError, buildCliCommands (plural), buildHostCommand, hostCmd field.

How is this tested?

Manual tests on versions 0.207.0 and 0.207.1

Unit tests in config/cli_token_source_test.go:

TestParseCliVersion — standard versions, patch versions, malformed output, empty string, missing prefix.
TestCliVersion_AtLeast — equal, higher/lower patch/minor/major, zero vs zero, zero vs nonzero.
TestBuildCliCommand — table-driven: version x config → expected command. Covers: host-only, account host, profile+new CLI (uses --profile), profile+old CLI (falls back to --host), profile-only+old CLI (nil), zero version (detection failed, falls back to --host), neither profile nor host (nil).
TestNewCliTokenSource — success with host, success with profile, CLI not found, neither profile nor host.
TestCliTokenSource_Token — success, CLI error, invalid JSON.

mihaimitrea-db · 2026-03-30T13:45:44Z

Range-diff: stack/force-refresh-flag (4cb4c26 -> 82d9599)

NEXT_CHANGELOG.md

@@ -6,5 +6,5 @@
  
 + * Generalize CLI token source into a progressive command list for forward-compatible flag support.
   * Normalize internal token sources on `auth.TokenSource` for proper context propagation ([#1577](https://github.com/databricks/databricks-sdk-go/pull/1577)).
-  * Bump golang.org/x/crypto from 0.21.0 to 0.45.0 in /examples/slog ([#1566](https://github.com/databricks/databricks-sdk-go/pull/1566)).
-  * Bump golang.org/x/net from 0.23.0 to 0.33.0 in /examples/slog ([#1127](https://github.com/databricks/databricks-sdk-go/pull/1127)).
\ No newline at end of file
+  * Fix `TestAzureGithubOIDCCredentials` hang caused by missing `HTTPTransport` stub: `EnsureResolved` now calls `resolveHostMetadata`, which makes a real network request when no transport is set ([#1550](https://github.com/databricks/databricks-sdk-go/pull/1550)).
+  * Bump golang.org/x/crypto from 0.21.0 to 0.45.0 in /examples/slog ([#1566](https://github.com/databricks/databricks-sdk-go/pull/1566)).
\ No newline at end of file

config/cli_token_source.go

@@ -18,13 +18,13 @@
 +	flag           string
 +	warningMessage string
 +}
-+
+ 
+-	// profileCmd uses --profile for token lookup. Nil when cfg.Profile is empty.
+-	profileCmd []string
 +var cliFeatureFlags = []cliFeatureFlag{
 +	{"--force-refresh", "Databricks CLI does not support --force-refresh flag. The CLI's token cache may provide stale tokens. Please upgrade your CLI to the latest version."},
 +}
- 
--	// profileCmd uses --profile for token lookup. Nil when cfg.Profile is empty.
--	profileCmd []string
++
 +const profileFlagWarning = "Databricks CLI does not support --profile flag. Falling back to --host. Please upgrade your CLI to the latest version."
  
 -	// hostCmd uses --host as a fallback for CLIs that predate --profile support.
@@ -83,7 +83,7 @@
 +				warning = cliFeatureFlags[i].warningMessage
 +			}
 +			commands = append(commands, cliCommand{args: args, warningMessage: warning})
-+			}
++		}
  	}
 -	if cfg.Host != "" {
 -		hostCmd = buildHostCommand(cliPath, cfg)

_{Reproduce locally: git range-diff f626fed..4cb4c26 67f79d8..82d9599 | Disable: git config gitstack.push-range-diff false}

mihaimitrea-db · 2026-03-30T13:56:02Z

Range-diff: stack/force-refresh-flag (82d9599 -> 68d45f4)

config/cli_token_source.go

@@ -18,13 +18,13 @@
 +	flag           string
 +	warningMessage string
 +}
- 
--	// profileCmd uses --profile for token lookup. Nil when cfg.Profile is empty.
--	profileCmd []string
++
 +var cliFeatureFlags = []cliFeatureFlag{
 +	{"--force-refresh", "Databricks CLI does not support --force-refresh flag. The CLI's token cache may provide stale tokens. Please upgrade your CLI to the latest version."},
 +}
-+
+ 
+-	// profileCmd uses --profile for token lookup. Nil when cfg.Profile is empty.
+-	profileCmd []string
 +const profileFlagWarning = "Databricks CLI does not support --profile flag. Falling back to --host. Please upgrade your CLI to the latest version."
  
 -	// hostCmd uses --host as a fallback for CLIs that predate --profile support.
@@ -143,10 +143,8 @@
 -		logger.Warnf(ctx, "Databricks CLI does not support --profile flag. Falling back to --host. Please upgrade your CLI to the latest version.")
 -	}
 -
--	if c.hostCmd != nil {
--		return c.execCliCommand(ctx, c.hostCmd)
--	}
--
- 	return nil, fmt.Errorf("no CLI command configured")
+-	return c.execCliCommand(ctx, c.hostCmd)
++	return nil, fmt.Errorf("cannot get access token: no CLI commands configured")
  }
- 
\ No newline at end of file
+ 
+ func (c *CliTokenSource) execCliCommand(ctx context.Context, args []string) (*oauth2.Token, error) {
\ No newline at end of file

_{Reproduce locally: git range-diff 67f79d8..82d9599 db4df21..68d45f4 | Disable: git config gitstack.push-range-diff false}

mihaimitrea-db · 2026-03-30T15:14:39Z

Range-diff: stack/force-refresh-flag (68d45f4 -> 4a5079f)

config/cli_token_source.go

@@ -116,6 +116,17 @@
  func (c *CliTokenSource) Token(ctx context.Context) (*oauth2.Token, error) {
 -	if c.forceCmd != nil {
 -		tok, err := c.execCliCommand(ctx, c.forceCmd)
+-		if err == nil {
+-			return tok, nil
+-		}
+-		if !isUnknownFlagError(err, "--force-refresh") && !isUnknownFlagError(err, "--profile") {
+-			return nil, err
+-		}
+-		logger.Warnf(ctx, "Databricks CLI does not support --force-refresh flag. The CLI's token cache may provide stale tokens. Please upgrade your CLI to the latest version.")
+-	}
+-
+-	if c.profileCmd != nil {
+-		tok, err := c.execCliCommand(ctx, c.profileCmd)
 +	for i := c.activeCommandIndex; i < len(c.commands); i++ {
 +		cmd := c.commands[i]
 +		tok, err := c.execCliCommand(ctx, cmd.args)
@@ -123,26 +134,18 @@
 +			c.activeCommandIndex = i
  			return tok, nil
  		}
--		if !isUnknownFlagError(err, "") {
+-		if !isUnknownFlagError(err, "--profile") {
 +		lastCommand := i == len(c.commands)-1
 +		if lastCommand || !isUnknownFlagError(err, "") {
  			return nil, err
  		}
--		logger.Warnf(ctx, "Databricks CLI does not support --force-refresh flag. The CLI's token cache may provide stale tokens. Please upgrade your CLI to the latest version.")
-+		logger.Warnf(ctx, cmd.warningMessage)
- 	}
--
--	if c.profileCmd != nil {
--		tok, err := c.execCliCommand(ctx, c.profileCmd)
--		if err == nil {
--			return tok, nil
--		}
--		if !isUnknownFlagError(err, "--profile") {
--			return nil, err
--		}
 -		logger.Warnf(ctx, "Databricks CLI does not support --profile flag. Falling back to --host. Please upgrade your CLI to the latest version.")
 -	}
 -
+-	if c.hostCmd == nil {
+-		return nil, fmt.Errorf("cannot get access token: no CLI commands available")
++		logger.Warnf(ctx, cmd.warningMessage)
+ 	}
 -	return c.execCliCommand(ctx, c.hostCmd)
 +	return nil, fmt.Errorf("cannot get access token: no CLI commands configured")
  }

config/cli_token_source_test.go

@@ -76,13 +76,13 @@
 -			gotForceCmd, gotProfileCmd, gotHostCmd := buildCliCommands(cliPath, tc.cfg)
 -			if !slices.Equal(gotForceCmd, tc.wantForceCmd) {
 -				t.Errorf("force cmd = %v, want %v", gotForceCmd, tc.wantForceCmd)
--			}
--			if !slices.Equal(gotProfileCmd, tc.wantProfileCmd) {
--				t.Errorf("profile cmd = %v, want %v", gotProfileCmd, tc.wantProfileCmd)
 +			got := buildCliCommands(cliPath, tc.cfg)
 +			if len(got) != len(tc.wantCommands) {
 +				t.Fatalf("got %d commands, want %d", len(got), len(tc.wantCommands))
  			}
+-			if !slices.Equal(gotProfileCmd, tc.wantProfileCmd) {
+-				t.Errorf("profile cmd = %v, want %v", gotProfileCmd, tc.wantProfileCmd)
+-			}
 -			if !slices.Equal(gotHostCmd, tc.wantHostCmd) {
 -				t.Errorf("host cmd = %v, want %v", gotHostCmd, tc.wantHostCmd)
 +			for i, want := range tc.wantCommands {
@@ -208,15 +208,15 @@
  	_, err := ts.Token(context.Background())
  	if err == nil {
  		t.Fatal("Token() error = nil, want error")
- 		t.Errorf("Token() error = %v, want error containing original auth failure", err)
  	}
  }
-+
+ 
+-func TestCliTokenSource_Token_NilHostCmdReturnsError(t *testing.T) {
 +func TestCliTokenSource_Token_ActiveCommandIndexPersists(t *testing.T) {
-+	if runtime.GOOS == "windows" {
-+		t.Skip("Skipping shell script test on Windows")
-+	}
-+
+ 	if runtime.GOOS == "windows" {
+ 		t.Skip("Skipping shell script test on Windows")
+ 	}
+ 
 +	expiry := time.Now().Add(1 * time.Hour).Format(time.RFC3339)
 +	validResponse, _ := json.Marshal(cliTokenResponse{
 +		AccessToken: "host-token",
@@ -224,18 +224,25 @@
 +		Expiry:      expiry,
 +	})
 +
-+	tempDir := t.TempDir()
-+
-+	forceScript := filepath.Join(tempDir, "force_cli.sh")
+ 	tempDir := t.TempDir()
+ 
+ 	forceScript := filepath.Join(tempDir, "force_cli.sh")
+-	if err := os.WriteFile(forceScript, []byte("#!/bin/sh\necho 'Error: unknown flag: --profile' >&2\nexit 1"), 0755); err != nil {
 +	if err := os.WriteFile(forceScript, []byte("#!/bin/sh\necho 'Error: unknown flag: --force-refresh' >&2\nexit 1"), 0755); err != nil {
-+		t.Fatalf("failed to create force script: %v", err)
-+	}
-+
+ 		t.Fatalf("failed to create force script: %v", err)
+ 	}
+ 
+-	profileScript := filepath.Join(tempDir, "profile_cli.sh")
+-	if err := os.WriteFile(profileScript, []byte("#!/bin/sh\necho 'Error: unknown flag: --profile' >&2\nexit 1"), 0755); err != nil {
+-		t.Fatalf("failed to create profile script: %v", err)
 +	hostScript := filepath.Join(tempDir, "host_cli.sh")
 +	if err := os.WriteFile(hostScript, []byte("#!/bin/sh\necho '"+string(validResponse)+"'"), 0755); err != nil {
 +		t.Fatalf("failed to create host script: %v", err)
-+	}
-+
+ 	}
+ 
+-	ts := &CliTokenSource{
+-		forceCmd:   []string{forceScript},
+-		profileCmd: []string{profileScript},
 +	ts := &CliTokenSource{commands: []cliCommand{
 +		{args: []string{forceScript}, warningMessage: "force-refresh not supported"},
 +		{args: []string{hostScript}},
@@ -245,13 +252,18 @@
 +	token, err := ts.Token(context.Background())
 +	if err != nil {
 +		t.Fatalf("first Token() error = %v", err)
-+	}
+ 	}
+-	_, err := ts.Token(context.Background())
+-	if err == nil {
+-		t.Fatal("Token() error = nil, want error")
 +	if token.AccessToken != "host-token" {
 +		t.Errorf("first AccessToken = %q, want %q", token.AccessToken, "host-token")
 +	}
 +	if ts.activeCommandIndex != 1 {
 +		t.Errorf("activeCommandIndex = %d, want 1", ts.activeCommandIndex)
-+	}
+ 	}
+-	if !strings.Contains(err.Error(), "no CLI commands available") {
+-		t.Errorf("Token() error = %v, want error containing %q", err, "no CLI commands available")
 +
 +	// Second call: starts at activeCommandIndex, skipping the force command.
 +	token, err = ts.Token(context.Background())
@@ -260,5 +272,5 @@
 +	}
 +	if token.AccessToken != "host-token" {
 +		t.Errorf("second AccessToken = %q, want %q", token.AccessToken, "host-token")
-+	}
-+}
\ No newline at end of file
+ 	}
+ }
\ No newline at end of file

_{Reproduce locally: git range-diff db4df21..68d45f4 97a1007..4a5079f | Disable: git config gitstack.push-range-diff false}

mihaimitrea-db · 2026-03-30T16:27:42Z

Range-diff: stack/force-refresh-flag (4a5079f -> 6f4fead)

config/cli_token_source.go

@@ -1,6 +1,13 @@
 diff --git a/config/cli_token_source.go b/config/cli_token_source.go
 --- a/config/cli_token_source.go
 +++ b/config/cli_token_source.go
+ 	"path/filepath"
+ 	"runtime"
+ 	"strings"
++	"sync/atomic"
+ 	"time"
+ 
+ 	"github.com/databricks/databricks-sdk-go/logger"
  	Expiry      string `json:"expiry"`
  }
  
@@ -42,7 +49,7 @@
 +// falling back progressively for older CLI versions that lack newer flags.
 +type CliTokenSource struct {
 +	commands           []cliCommand
-+	activeCommandIndex int
++	activeCommandIndex atomic.Int32
  }
  
  func NewCliTokenSource(cfg *Config) (*CliTokenSource, error) {
@@ -127,11 +134,11 @@
 -
 -	if c.profileCmd != nil {
 -		tok, err := c.execCliCommand(ctx, c.profileCmd)
-+	for i := c.activeCommandIndex; i < len(c.commands); i++ {
++	for i := int(c.activeCommandIndex.Load()); i < len(c.commands); i++ {
 +		cmd := c.commands[i]
 +		tok, err := c.execCliCommand(ctx, cmd.args)
  		if err == nil {
-+			c.activeCommandIndex = i
++			c.activeCommandIndex.Store(int32(i))
  			return tok, nil
  		}
 -		if !isUnknownFlagError(err, "--profile") {

config/cli_token_source_test.go

@@ -259,8 +259,8 @@
 +	if token.AccessToken != "host-token" {
 +		t.Errorf("first AccessToken = %q, want %q", token.AccessToken, "host-token")
 +	}
-+	if ts.activeCommandIndex != 1 {
-+		t.Errorf("activeCommandIndex = %d, want 1", ts.activeCommandIndex)
++	if ts.activeCommandIndex.Load() != 1 {
++		t.Errorf("activeCommandIndex = %d, want 1", ts.activeCommandIndex.Load())
  	}
 -	if !strings.Contains(err.Error(), "no CLI commands available") {
 -		t.Errorf("Token() error = %v, want error containing %q", err, "no CLI commands available")

_{Reproduce locally: git range-diff 97a1007..4a5079f 69a7c95..6f4fead | Disable: git config gitstack.push-range-diff false}

mihaimitrea-db · 2026-03-31T08:07:49Z

Range-diff: stack/force-refresh-flag (6f4fead -> 2218270)

config/cli_token_source.go

@@ -11,43 +11,29 @@
  	Expiry      string `json:"expiry"`
  }
  
--// CliTokenSource fetches OAuth tokens by shelling out to the Databricks CLI.
++// cliCommand is a single CLI invocation with an associated warning message
++// that is logged when this command fails and we fall back to the next one.
++type cliCommand struct {
++	args           []string
++	warningMessage string
++}
++
+ // CliTokenSource fetches OAuth tokens by shelling out to the Databricks CLI.
 -// Commands are tried in order: forceCmd -> profileCmd -> hostCmd, progressively
 -// falling back to simpler invocations for older CLI versions.
--type CliTokenSource struct {
++// It holds a list of commands ordered from most feature-rich to simplest,
++// falling back progressively for older CLI versions that lack newer flags.
+ type CliTokenSource struct {
 -	// forceCmd uses --profile with --force-refresh to bypass the CLI's token cache.
 -	// Nil when cfg.Profile is empty (--force-refresh requires --profile support).
 -	forceCmd []string
-+// cliFeatureFlag defines a CLI feature flag and the warning to log when
-+// falling back because the CLI does not support it. Ordered newest-first:
-+// commands are built by progressively stripping these flags for older CLIs.
-+type cliFeatureFlag struct {
-+	flag           string
-+	warningMessage string
-+}
-+
-+var cliFeatureFlags = []cliFeatureFlag{
-+	{"--force-refresh", "Databricks CLI does not support --force-refresh flag. The CLI's token cache may provide stale tokens. Please upgrade your CLI to the latest version."},
-+}
- 
+-
 -	// profileCmd uses --profile for token lookup. Nil when cfg.Profile is empty.
 -	profileCmd []string
-+const profileFlagWarning = "Databricks CLI does not support --profile flag. Falling back to --host. Please upgrade your CLI to the latest version."
- 
+-
 -	// hostCmd uses --host as a fallback for CLIs that predate --profile support.
 -	// Nil when cfg.Host is empty.
 -	hostCmd []string
-+// cliCommand is a single CLI invocation with an associated warning message
-+// that is logged when this command fails and we fall back to the next one.
-+type cliCommand struct {
-+	args           []string
-+	warningMessage string
-+}
-+
-+// CliTokenSource fetches OAuth tokens by shelling out to the Databricks CLI.
-+// It holds a list of commands ordered from most feature-rich to simplest,
-+// falling back progressively for older CLI versions that lack newer flags.
-+type CliTokenSource struct {
 +	commands           []cliCommand
 +	activeCommandIndex atomic.Int32
  }
@@ -69,48 +55,47 @@
 -func buildCliCommands(cliPath string, cfg *Config) ([]string, []string, []string) {
 -	var forceCmd, profileCmd, hostCmd []string
 +// buildCliCommands constructs the list of CLI commands for fetching an auth
-+// token, ordered from most feature-rich to simplest. When cfg.Profile is set,
-+// commands include feature flags from [cliFeatureFlags] (stripped progressively)
-+// plus a --host fallback when host is available. When cfg.Profile is empty,
-+// only --host is returned — the CLI must support --profile before any feature
-+// flags can be used (monotonic feature assumption).
++// token, ordered from most feature-rich to simplest. The order defines the
++// fallback chain: when a command fails with an unknown flag error, the next
++// one is tried.
 +func buildCliCommands(cliPath string, cfg *Config) []cliCommand {
 +	var commands []cliCommand
  	if cfg.Profile != "" {
 -		profileCmd = []string{cliPath, "auth", "token", "--profile", cfg.Profile}
 -		forceCmd = append(profileCmd, "--force-refresh")
-+		baseArgs := []string{cliPath, "auth", "token", "--profile", cfg.Profile}
-+		for i := 0; i <= len(cliFeatureFlags); i++ {
-+			args := append([]string{}, baseArgs...)
-+			for _, f := range cliFeatureFlags[i:] {
-+				args = append(args, f.flag)
-+			}
-+			warning := profileFlagWarning
-+			if i < len(cliFeatureFlags) {
-+				warning = cliFeatureFlags[i].warningMessage
-+			}
-+			commands = append(commands, cliCommand{args: args, warningMessage: warning})
-+		}
- 	}
+-	}
 -	if cfg.Host != "" {
 -		hostCmd = buildHostCommand(cliPath, cfg)
-+	hostArgs := buildHostCommand(cliPath, cfg)
-+	if hostArgs != nil {
-+		commands = append(commands, cliCommand{args: hostArgs})
- 	}
+-	}
 -	return forceCmd, profileCmd, hostCmd
++		commands = append(commands, cliCommand{
++			args:           []string{cliPath, "auth", "token", "--profile", cfg.Profile, "--force-refresh"},
++			warningMessage: "Databricks CLI does not support --force-refresh flag. The CLI's token cache may provide stale tokens. Please upgrade your CLI to the latest version.",
++		})
++		commands = append(commands, cliCommand{
++			args:           []string{cliPath, "auth", "token", "--profile", cfg.Profile},
++			warningMessage: "Databricks CLI does not support --profile flag. Falling back to --host. Please upgrade your CLI to the latest version.",
++		})
++	}
++	commands = appendHostCommand(commands, cliPath, cfg)
 +	return commands
  }
  
- // buildHostCommand constructs the legacy --host based CLI command.
-+// Returns nil when cfg.Host is empty.
- func buildHostCommand(cliPath string, cfg *Config) []string {
+-// buildHostCommand constructs the legacy --host based CLI command.
+-func buildHostCommand(cliPath string, cfg *Config) []string {
+-	cmd := []string{cliPath, "auth", "token", "--host", cfg.Host}
++func appendHostCommand(commands []cliCommand, cliPath string, cfg *Config) []cliCommand {
 +	if cfg.Host == "" {
-+		return nil
++		return commands
 +	}
- 	cmd := []string{cliPath, "auth", "token", "--host", cfg.Host}
++	args := []string{cliPath, "auth", "token", "--host", cfg.Host}
  	switch cfg.HostType() {
  	case AccountHost:
+-		cmd = append(cmd, "--account-id", cfg.AccountID)
++		args = append(args, "--account-id", cfg.AccountID)
+ 	}
+-	return cmd
++	return append(commands, cliCommand{args: args})
  }
  
  // Token fetches an OAuth token by shelling out to the Databricks CLI.

_{Reproduce locally: git range-diff 69a7c95..6f4fead 69a7c95..2218270 | Disable: git config gitstack.push-range-diff false}

mihaimitrea-db · 2026-03-31T09:06:40Z

Range-diff: stack/force-refresh-flag (2218270 -> 76e74ca)

config/cli_token_source.go

@@ -15,6 +15,7 @@
 +// that is logged when this command fails and we fall back to the next one.
 +type cliCommand struct {
 +	args           []string
++	flags          []string
 +	warningMessage string
 +}
 +
@@ -70,10 +71,12 @@
 -	return forceCmd, profileCmd, hostCmd
 +		commands = append(commands, cliCommand{
 +			args:           []string{cliPath, "auth", "token", "--profile", cfg.Profile, "--force-refresh"},
++			flags:          []string{"--force-refresh", "--profile"},
 +			warningMessage: "Databricks CLI does not support --force-refresh flag. The CLI's token cache may provide stale tokens. Please upgrade your CLI to the latest version.",
 +		})
 +		commands = append(commands, cliCommand{
 +			args:           []string{cliPath, "auth", "token", "--profile", cfg.Profile},
++			flags:          []string{"--profile"},
 +			warningMessage: "Databricks CLI does not support --profile flag. Falling back to --host. Please upgrade your CLI to the latest version.",
 +		})
 +	}
@@ -108,17 +111,6 @@
  func (c *CliTokenSource) Token(ctx context.Context) (*oauth2.Token, error) {
 -	if c.forceCmd != nil {
 -		tok, err := c.execCliCommand(ctx, c.forceCmd)
--		if err == nil {
--			return tok, nil
--		}
--		if !isUnknownFlagError(err, "--force-refresh") && !isUnknownFlagError(err, "--profile") {
--			return nil, err
--		}
--		logger.Warnf(ctx, "Databricks CLI does not support --force-refresh flag. The CLI's token cache may provide stale tokens. Please upgrade your CLI to the latest version.")
--	}
--
--	if c.profileCmd != nil {
--		tok, err := c.execCliCommand(ctx, c.profileCmd)
 +	for i := int(c.activeCommandIndex.Load()); i < len(c.commands); i++ {
 +		cmd := c.commands[i]
 +		tok, err := c.execCliCommand(ctx, cmd.args)
@@ -126,20 +118,52 @@
 +			c.activeCommandIndex.Store(int32(i))
  			return tok, nil
  		}
--		if !isUnknownFlagError(err, "--profile") {
+-		if !isUnknownFlagError(err, "--force-refresh") && !isUnknownFlagError(err, "--profile") {
 +		lastCommand := i == len(c.commands)-1
-+		if lastCommand || !isUnknownFlagError(err, "") {
++		if lastCommand || !isUnknownFlagError(err, cmd.flags) {
  			return nil, err
  		}
+-		logger.Warnf(ctx, "Databricks CLI does not support --force-refresh flag. The CLI's token cache may provide stale tokens. Please upgrade your CLI to the latest version.")
++		logger.Warnf(ctx, cmd.warningMessage)
+ 	}
+-
+-	if c.profileCmd != nil {
+-		tok, err := c.execCliCommand(ctx, c.profileCmd)
+-		if err == nil {
+-			return tok, nil
+-		}
+-		if !isUnknownFlagError(err, "--profile") {
+-			return nil, err
+-		}
 -		logger.Warnf(ctx, "Databricks CLI does not support --profile flag. Falling back to --host. Please upgrade your CLI to the latest version.")
 -	}
 -
 -	if c.hostCmd == nil {
 -		return nil, fmt.Errorf("cannot get access token: no CLI commands available")
-+		logger.Warnf(ctx, cmd.warningMessage)
- 	}
+-	}
 -	return c.execCliCommand(ctx, c.hostCmd)
 +	return nil, fmt.Errorf("cannot get access token: no CLI commands configured")
  }
  
- func (c *CliTokenSource) execCliCommand(ctx context.Context, args []string) (*oauth2.Token, error) {
\ No newline at end of file
+ func (c *CliTokenSource) execCliCommand(ctx context.Context, args []string) (*oauth2.Token, error) {
+ }
+ 
+ // isUnknownFlagError returns true if the error indicates the CLI does not
+-// recognize a flag. Pass a specific flag (e.g. "--profile") to check for that
+-// flag, or pass "" to match any "unknown flag:" error.
+-func isUnknownFlagError(err error, flag string) bool {
+-	if flag == "" {
+-		return strings.Contains(err.Error(), "unknown flag:")
++// recognize one of the given flags.
++func isUnknownFlagError(err error, flags []string) bool {
++	msg := err.Error()
++	for _, flag := range flags {
++		if strings.Contains(msg, "unknown flag: "+flag) {
++			return true
++		}
+ 	}
+-	return strings.Contains(err.Error(), "unknown flag: "+flag)
++	return false
+ }
+ 
+ // parseExpiry parses an expiry time string in multiple formats for cross-SDK compatibility.
\ No newline at end of file

config/cli_token_source_test.go

@@ -121,7 +121,7 @@
 -		profileCmd: []string{profileScript},
 -	}
 +	ts := &CliTokenSource{commands: []cliCommand{
-+		{args: []string{forceScript}, warningMessage: "force-refresh not supported"},
++		{args: []string{forceScript}, flags: []string{"--force-refresh"}, warningMessage: "force-refresh not supported"},
 +		{args: []string{profileScript}},
 +	}}
  	token, err := ts.Token(context.Background())
@@ -135,7 +135,7 @@
 -		hostCmd:    []string{hostScript},
 -	}
 +	ts := &CliTokenSource{commands: []cliCommand{
-+		{args: []string{profileScript}, warningMessage: "profile not supported"},
++		{args: []string{profileScript}, flags: []string{"--profile"}, warningMessage: "profile not supported"},
 +		{args: []string{hostScript}},
 +	}}
  	token, err := ts.Token(context.Background())
@@ -171,8 +171,8 @@
 -		hostCmd:    []string{hostScript},
 -	}
 +	ts := &CliTokenSource{commands: []cliCommand{
-+		{args: []string{forceScript}, warningMessage: "force-refresh not supported"},
-+		{args: []string{profileScript}, warningMessage: "profile not supported"},
++		{args: []string{forceScript}, flags: []string{"--force-refresh", "--profile"}, warningMessage: "force-refresh not supported"},
++		{args: []string{profileScript}, flags: []string{"--profile"}, warningMessage: "profile not supported"},
 +		{args: []string{hostScript}},
 +	}}
  	token, err := ts.Token(context.Background())
@@ -201,8 +201,8 @@
 -		hostCmd:    []string{hostScript},
 -	}
 +	ts := &CliTokenSource{commands: []cliCommand{
-+		{args: []string{forceScript}, warningMessage: "force-refresh not supported"},
-+		{args: []string{profileScript}, warningMessage: "profile not supported"},
++		{args: []string{forceScript}, flags: []string{"--force-refresh", "--profile"}, warningMessage: "force-refresh not supported"},
++		{args: []string{profileScript}, flags: []string{"--profile"}, warningMessage: "profile not supported"},
 +		{args: []string{hostScript}},
 +	}}
  	_, err := ts.Token(context.Background())
@@ -244,7 +244,7 @@
 -		forceCmd:   []string{forceScript},
 -		profileCmd: []string{profileScript},
 +	ts := &CliTokenSource{commands: []cliCommand{
-+		{args: []string{forceScript}, warningMessage: "force-refresh not supported"},
++		{args: []string{forceScript}, flags: []string{"--force-refresh"}, warningMessage: "force-refresh not supported"},
 +		{args: []string{hostScript}},
 +	}}
 +

_{Reproduce locally: git range-diff 69a7c95..2218270 69a7c95..76e74ca | Disable: git config gitstack.push-range-diff false}

renaudhartert-db · 2026-03-31T14:55:08Z

 		var exitErr *exec.ExitError
 		if errors.As(err, &exitErr) {
-			return nil, fmt.Errorf("cannot get access token: %s", strings.TrimSpace(string(exitErr.Stderr)))
+			return nil, fmt.Errorf("cannot get access token: %q", strings.TrimSpace(string(exitErr.Stderr)))


Sorry, what I meant is that we are losing the exec.ExitError type. The new type that we have introduced does not solve that.

Sign in to view

+// Token fetches an OAuth token by shelling out to the Databricks CLI.
+// If the working command has already been identified, it is called directly.
+// Otherwise, [probeAndExec] tries each command in order to find one that works.
+func (c *CliTokenSource) Token(ctx context.Context) (*oauth2.Token, error) {


renaudhartert-db · 2026-04-01T11:18:58Z

-		return c.execCliCommand(ctx, c.hostCmd)
+// probeAndExec walks the command list from most-featured to simplest, looking
+// for a CLI command that succeeds. When a command fails with "unknown flag" for
+// one of its [cliCommand.usedFlags], it logs a warning and tries the next.


Suggested change

// one of its [cliCommand.usedFlags], it logs a warning and tries the next.

// one of its [cliCommand.usedFlags], it logs a warning and tries the next. The "counter" is not incremented if the command fails for any other reason.

renaudhartert-db · 2026-04-01T11:26:39Z

 		var exitErr *exec.ExitError
 		if errors.As(err, &exitErr) {
-			return nil, fmt.Errorf("cannot get access token: %s", strings.TrimSpace(string(exitErr.Stderr)))
+			return nil, fmt.Errorf("cannot get access token: %w",


From offline discussion: we are loosing the exec.ExitError type. I don't have an opinion on whether we should keep it or not. Though, if we don't let's be explicit about the rationale.

mihaimitrea-db · 2026-04-07T09:18:51Z

Range-diff: stack/force-refresh-flag (79ac794 -> a19dab0)

NEXT_CHANGELOG.md

@@ -1,10 +1,10 @@
 diff --git a/NEXT_CHANGELOG.md b/NEXT_CHANGELOG.md
 --- a/NEXT_CHANGELOG.md
 +++ b/NEXT_CHANGELOG.md
- 
  ### Internal Changes
  
+  * Pass `--force-refresh` to Databricks CLI `auth token` command to bypass the CLI's internal token cache.
 + * Generalize CLI token source into a progressive command list for forward-compatible flag support.
+  * Use resolved host type from host metadata in `HostType()` method, falling back to URL pattern matching when metadata is unavailable.
   * Normalize internal token sources on `auth.TokenSource` for proper context propagation ([#1577](https://github.com/databricks/databricks-sdk-go/pull/1577)).
-  * Fix `TestAzureGithubOIDCCredentials` hang caused by missing `HTTPTransport` stub: `EnsureResolved` now calls `resolveHostMetadata`, which makes a real network request when no transport is set ([#1550](https://github.com/databricks/databricks-sdk-go/pull/1550)).
-  * Bump golang.org/x/crypto from 0.21.0 to 0.45.0 in /examples/slog ([#1566](https://github.com/databricks/databricks-sdk-go/pull/1566)).
\ No newline at end of file
+  * Fix `TestAzureGithubOIDCCredentials` hang caused by missing `HTTPTransport` stub: `EnsureResolved` now calls `resolveHostMetadata`, which makes a real network request when no transport is set ([#1550](https://github.com/databricks/databricks-sdk-go/pull/1550)).
\ No newline at end of file

config/cli_token_source.go

@@ -11,24 +11,6 @@
  	Expiry      string `json:"expiry"`
  }
  
--// CliTokenSource fetches OAuth tokens by shelling out to the Databricks CLI.
--// Commands are tried in order: forceCmd -> profileCmd -> hostCmd, progressively
--// falling back to simpler invocations for older CLI versions.
--type CliTokenSource struct {
--	// forceCmd uses --profile with --force-refresh to bypass the CLI's token cache.
--	// Nil when cfg.Profile is empty (--force-refresh requires --profile support).
--	forceCmd []string
-+// cliError wraps stderr output from a failed CLI invocation.
-+type cliError struct {
-+	stderr string
-+}
- 
--	// profileCmd uses --profile for token lookup. Nil when cfg.Profile is empty.
--	profileCmd []string
-+func (e *cliError) Error() string {
-+	return e.stderr
-+}
-+
 +// cliCommand is a single CLI invocation with an associated warning message
 +// that is logged when this command fails and we fall back to the next one.
 +type cliCommand struct {
@@ -38,14 +20,26 @@
 +	usedFlags      []string
 +	warningMessage string
 +}
- 
++
+ // CliTokenSource fetches OAuth tokens by shelling out to the Databricks CLI.
+-// Commands are tried in order: forceCmd -> profileCmd -> hostCmd, progressively
+-// falling back to simpler invocations for older CLI versions.
++// It holds a list of commands ordered from most feature-rich to simplest,
++// falling back progressively for older CLI versions that lack newer flags.
+ type CliTokenSource struct {
+-	// forceCmd appends --force-refresh to the base command (profileCmd when a
+-	// profile is configured, hostCmd otherwise) to bypass the CLI's token cache.
+-	// Nil when neither profile nor host is set.
+-	// CLI support: >= v0.296.0 (databricks/cli#4767).
+-	forceCmd []string
+-
+-	// profileCmd uses --profile for token lookup. Nil when cfg.Profile is empty.
+-	// CLI support: >= v0.207.1 (databricks/cli#855).
+-	profileCmd []string
+-
 -	// hostCmd uses --host as a fallback for CLIs that predate --profile support.
 -	// Nil when cfg.Host is empty.
 -	hostCmd []string
-+// CliTokenSource fetches OAuth tokens by shelling out to the Databricks CLI.
-+// It holds a list of commands ordered from most feature-rich to simplest,
-+// falling back progressively for older CLI versions that lack newer flags.
-+type CliTokenSource struct {
 +	commands []cliCommand
 +	// activeCommandIndex is the index of the CLI command known to work, or -1
 +	// if not yet resolved. Once resolved it never changes — older CLIs don't
@@ -71,21 +65,24 @@
  }
  
 -// buildCliCommands constructs the CLI commands for fetching an auth token.
--// When cfg.Profile is set, three commands are built: a --force-refresh variant,
--// a plain --profile variant, and (when host is available) a --host fallback.
--// When cfg.Profile is empty, only --host is returned — the CLI must support
--// --profile before --force-refresh can be used (monotonic feature assumption).
+-// When cfg.Profile is set, three commands are built: a --force-refresh variant
+-// (based on profileCmd), a plain --profile variant, and (when host is available)
+-// a --host fallback. When cfg.Profile is empty, --force-refresh is based on the
+-// --host command instead.
 -func buildCliCommands(cliPath string, cfg *Config) ([]string, []string, []string) {
 -	var forceCmd, profileCmd, hostCmd []string
 +// buildCliCommands constructs the list of CLI commands for fetching an auth
 +// token, ordered from most feature-rich to simplest. The order defines the
 +// fallback chain: when a command fails with an unknown flag error, the next
 +// one is tried.
++//
++// When cfg.Profile is set, --force-refresh is based on the --profile command.
++// When cfg.Profile is empty, --force-refresh is based on the --host command
++// instead, so host-only configurations still benefit from cache bypass.
 +func buildCliCommands(cliPath string, cfg *Config) []cliCommand {
 +	var commands []cliCommand
  	if cfg.Profile != "" {
 -		profileCmd = []string{cliPath, "auth", "token", "--profile", cfg.Profile}
--		forceCmd = append(profileCmd, "--force-refresh")
 +		commands = append(commands, cliCommand{
 +			args:           []string{cliPath, "auth", "token", "--profile", cfg.Profile, "--force-refresh"},
 +			usedFlags:      []string{"--force-refresh", "--profile"},
@@ -100,6 +97,11 @@
  	if cfg.Host != "" {
 -		hostCmd = buildHostCommand(cliPath, cfg)
 -	}
+-	if profileCmd != nil {
+-		forceCmd = append(profileCmd, "--force-refresh")
+-	} else if hostCmd != nil {
+-		forceCmd = append(hostCmd, "--force-refresh")
+-	}
 -	return forceCmd, profileCmd, hostCmd
 -}
 -
@@ -109,12 +111,20 @@
 -	switch cfg.HostType() {
 -	case AccountHost:
 -		cmd = append(cmd, "--account-id", cfg.AccountID)
-+		args := []string{cliPath, "auth", "token", "--host", cfg.Host}
++		hostArgs := []string{cliPath, "auth", "token", "--host", cfg.Host}
 +		switch cfg.HostType() {
 +		case AccountHost:
-+			args = append(args, "--account-id", cfg.AccountID)
++			hostArgs = append(hostArgs, "--account-id", cfg.AccountID)
 +		}
-+		commands = append(commands, cliCommand{args: args})
++		if cfg.Profile == "" {
++			forceArgs := append(hostArgs[:len(hostArgs):len(hostArgs)], "--force-refresh")
++			commands = append(commands, cliCommand{
++				args:           forceArgs,
++				usedFlags:      []string{"--force-refresh"},
++				warningMessage: "Databricks CLI does not support --force-refresh flag. The CLI's token cache may provide stale tokens. Please upgrade your CLI to the latest version.",
++			})
++		}
++		commands = append(commands, cliCommand{args: hostArgs})
  	}
 -	return cmd
 +	return commands
@@ -124,8 +134,6 @@
 -// Commands are tried in order: forceCmd -> profileCmd -> hostCmd, skipping nil
 -// entries. Each command falls through to the next on "unknown flag" errors,
 -// logging a warning about the unsupported feature.
-+// If the working command has already been identified, it is called directly.
-+// Otherwise, [probeAndExec] tries each command in order to find one that works.
  func (c *CliTokenSource) Token(ctx context.Context) (*oauth2.Token, error) {
 -	if c.forceCmd != nil {
 -		tok, err := c.execCliCommand(ctx, c.forceCmd)
@@ -136,6 +144,8 @@
 -			return nil, err
 -		}
 -		logger.Warnf(ctx, "Databricks CLI does not support --force-refresh flag. The CLI's token cache may provide stale tokens. Please upgrade your CLI to the latest version.")
++	// If the working command has already been identified, call it directly.
++	// Otherwise, probe each command in order to find one that works.
 +	if idx := int(c.activeCommandIndex.Load()); idx >= 0 {
 +		return c.execCliCommand(ctx, c.commands[idx].args)
  	}
@@ -146,7 +156,8 @@
 -		tok, err := c.execCliCommand(ctx, c.profileCmd)
 +// probeAndExec walks the command list from most-featured to simplest, looking
 +// for a CLI command that succeeds. When a command fails with "unknown flag" for
-+// one of its [cliCommand.usedFlags], it logs a warning and tries the next.
++// one of its [cliCommand.usedFlags], it logs a warning and tries the next. Any
++// other error stops probing immediately and is returned to the caller.
 +// On success, [activeCommandIndex] is stored so future calls skip probing.
 +func (c *CliTokenSource) probeAndExec(ctx context.Context) (*oauth2.Token, error) {
 +	for i := range c.commands {
@@ -177,8 +188,13 @@
  		var exitErr *exec.ExitError
  		if errors.As(err, &exitErr) {
 -			return nil, fmt.Errorf("cannot get access token: %q", strings.TrimSpace(string(exitErr.Stderr)))
-+			return nil, fmt.Errorf("cannot get access token: %w",
-+				&cliError{stderr: strings.TrimSpace(string(exitErr.Stderr))})
++			// Format with %q: the CLI's stderr can be multi-line (includes usage
++			// text on unknown-flag errors) and quoting keeps it on one line so
++			// log aggregators don't split a single error across multiple entries.
++			// We intentionally discard exec.ExitError — the stderr text is the
++			// CLI's error contract; exit codes and process state are not useful.
++			return nil, fmt.Errorf("cannot get access token: %q",
++				strings.TrimSpace(string(exitErr.Stderr)))
  		}
  		return nil, fmt.Errorf("cannot get access token: %w", err)
  	}
@@ -186,24 +202,18 @@
  }
  
 -// isUnknownFlagError returns true if the error indicates the CLI does not
--// recognize a flag. Pass a specific flag (e.g. "--profile") to check for that
--// flag, or pass "" to match any "unknown flag:" error.
+-// recognize the given flag (e.g. "--profile", "--force-refresh").
 -func isUnknownFlagError(err error, flag string) bool {
--	if flag == "" {
--		return strings.Contains(err.Error(), "unknown flag:")
-+// isUnknownFlagError returns true if the error wraps a [cliError] whose stderr
-+// indicates the CLI does not recognize one of the given flags.
+-	return strings.Contains(err.Error(), "unknown flag: "+flag)
++// isUnknownFlagError returns true if the error message indicates the CLI does
++// not recognize one of the given flags.
 +func isUnknownFlagError(err error, flags []string) bool {
-+	var cliErr *cliError
-+	if !errors.As(err, &cliErr) {
-+		return false
-+	}
++	msg := err.Error()
 +	for _, flag := range flags {
-+		if strings.Contains(cliErr.stderr, "unknown flag: "+flag) {
++		if strings.Contains(msg, "unknown flag: "+flag) {
 +			return true
 +		}
- 	}
--	return strings.Contains(err.Error(), "unknown flag: "+flag)
++	}
 +	return false
  }

config/cli_token_source_test.go

@@ -14,22 +14,26 @@
 +		wantCommands [][]string
  	}{
  		{
--			name:        "workspace host only",
--			cfg:         &Config{Host: host},
--			wantHostCmd: []string{cliPath, "auth", "token", "--host", host},
-+			name: "workspace host only",
+-			name:         "workspace host only — force-refresh based on host",
+-			cfg:          &Config{Host: host},
+-			wantForceCmd: []string{cliPath, "auth", "token", "--host", host, "--force-refresh"},
+-			wantHostCmd:  []string{cliPath, "auth", "token", "--host", host},
++			name: "workspace host only — force-refresh based on host",
 +			cfg:  &Config{Host: host},
 +			wantCommands: [][]string{
++				{cliPath, "auth", "token", "--host", host, "--force-refresh"},
 +				{cliPath, "auth", "token", "--host", host},
 +			},
  		},
  		{
--			name:        "account host only",
--			cfg:         &Config{Host: accountHost, AccountID: accountID},
--			wantHostCmd: []string{cliPath, "auth", "token", "--host", accountHost, "--account-id", accountID},
-+			name: "account host only",
+-			name:         "account host only — force-refresh based on host with account-id",
+-			cfg:          &Config{Host: accountHost, AccountID: accountID},
+-			wantForceCmd: []string{cliPath, "auth", "token", "--host", accountHost, "--account-id", accountID, "--force-refresh"},
+-			wantHostCmd:  []string{cliPath, "auth", "token", "--host", accountHost, "--account-id", accountID},
++			name: "account host only — force-refresh based on host with account-id",
 +			cfg:  &Config{Host: accountHost, AccountID: accountID},
 +			wantCommands: [][]string{
++				{cliPath, "auth", "token", "--host", accountHost, "--account-id", accountID, "--force-refresh"},
 +				{cliPath, "auth", "token", "--host", accountHost, "--account-id", accountID},
 +			},
  		},
@@ -38,18 +42,20 @@
  				AccountID:   accountID,
  				WorkspaceID: workspaceID,
  			},
--			wantHostCmd: []string{cliPath, "auth", "token", "--host", unifiedHost},
+-			wantForceCmd: []string{cliPath, "auth", "token", "--host", unifiedHost, "--force-refresh"},
+-			wantHostCmd:  []string{cliPath, "auth", "token", "--host", unifiedHost},
 +			wantCommands: [][]string{
++				{cliPath, "auth", "token", "--host", unifiedHost, "--force-refresh"},
 +				{cliPath, "auth", "token", "--host", unifiedHost},
 +			},
  		},
  		{
--			name:           "profile with host — all three commands",
+-			name:           "profile with host — force-refresh based on profile",
 -			cfg:            &Config{Profile: "my-profile", Host: host},
 -			wantForceCmd:   []string{cliPath, "auth", "token", "--profile", "my-profile", "--force-refresh"},
 -			wantProfileCmd: []string{cliPath, "auth", "token", "--profile", "my-profile"},
 -			wantHostCmd:    []string{cliPath, "auth", "token", "--host", host},
-+			name: "profile with host — all three commands",
++			name: "profile with host — force-refresh based on profile",
 +			cfg:  &Config{Profile: "my-profile", Host: host},
 +			wantCommands: [][]string{
 +				{cliPath, "auth", "token", "--profile", "my-profile", "--force-refresh"},
@@ -136,110 +142,70 @@
  			token, err := ts.Token(context.Background())
  
  			if tc.wantErrMsg != "" {
- 		t.Fatalf("failed to create profile script: %v", err)
+ 			wantErrMsg:    "databricks OAuth is not configured",
+ 		},
+ 		{
+-			name:          "nil hostCmd after profile failure returns error",
++			name:          "all commands fail with unknown flag — last error returned",
+ 			forceScript:   unknownProfile,
+ 			profileScript: unknownProfile,
+-			wantErrMsg:    "no CLI commands available",
++			wantErrMsg:    "unknown flag: --profile",
+ 		},
  	}
  
--	ts := &CliTokenSource{
--		forceCmd:   []string{forceScript},
--		profileCmd: []string{profileScript},
--	}
-+	ts := newTestCliTokenSource([]cliCommand{
-+		{args: []string{forceScript}, usedFlags: []string{"--force-refresh"}, warningMessage: "force-refresh not supported"},
-+		{args: []string{profileScript}},
-+	})
- 	token, err := ts.Token(context.Background())
- 	if err != nil {
- 		t.Fatalf("Token() error = %v, want fallback to profileCmd to succeed", err)
- 		t.Fatalf("failed to create host script: %v", err)
- 	}
+ 	for _, tc := range testCases {
+ 		t.Run(tc.name, func(t *testing.T) {
+ 			tempDir := t.TempDir()
+-			var ts CliTokenSource
++			var commands []cliCommand
  
--	ts := &CliTokenSource{
--		profileCmd: []string{profileScript},
--		hostCmd:    []string{hostScript},
--	}
-+	ts := newTestCliTokenSource([]cliCommand{
-+		{args: []string{profileScript}, usedFlags: []string{"--profile"}, warningMessage: "profile not supported"},
-+		{args: []string{hostScript}},
-+	})
- 	token, err := ts.Token(context.Background())
- 	if err != nil {
- 		t.Fatalf("Token() error = %v, want fallback to hostCmd to succeed", err)
- 	}
- }
+ 			if tc.forceScript != "" {
+ 				path := filepath.Join(tempDir, "force_cli.sh")
+ 				if err := os.WriteFile(path, []byte(tc.forceScript), 0755); err != nil {
+ 					t.Fatalf("failed to create force script: %v", err)
+ 				}
+-				ts.forceCmd = []string{path}
++				commands = append(commands, cliCommand{
++					args:           []string{path},
++					usedFlags:      []string{"--force-refresh", "--profile"},
++					warningMessage: "force-refresh not supported",
++				})
+ 			}
+ 			if tc.profileScript != "" {
+ 				path := filepath.Join(tempDir, "profile_cli.sh")
+ 				if err := os.WriteFile(path, []byte(tc.profileScript), 0755); err != nil {
+ 					t.Fatalf("failed to create profile script: %v", err)
+ 				}
+-				ts.profileCmd = []string{path}
++				commands = append(commands, cliCommand{
++					args:           []string{path},
++					usedFlags:      []string{"--profile"},
++					warningMessage: "profile not supported",
++				})
+ 			}
+ 			if tc.hostScript != "" {
+ 				path := filepath.Join(tempDir, "host_cli.sh")
+ 				if err := os.WriteFile(path, []byte(tc.hostScript), 0755); err != nil {
+ 					t.Fatalf("failed to create host script: %v", err)
+ 				}
+-				ts.hostCmd = []string{path}
++				commands = append(commands, cliCommand{args: []string{path}})
+ 			}
  
--func TestCliTokenSource_Token_ForceRefreshFallbackToHostOnProfileError(t *testing.T) {
-+func TestCliTokenSource_Token_FullFallbackChain(t *testing.T) {
- 	if runtime.GOOS == "windows" {
- 		t.Skip("Skipping shell script test on Windows")
- 	}
- 
- 	tempDir := t.TempDir()
- 
--	// forceCmd fails with --profile unknown (very old CLI).
- 	forceScript := filepath.Join(tempDir, "force_cli.sh")
- 	if err := os.WriteFile(forceScript, []byte("#!/bin/sh\necho 'Error: unknown flag: --profile' >&2\nexit 1"), 0755); err != nil {
- 		t.Fatalf("failed to create force script: %v", err)
- 	}
- 
--	// profileCmd also fails with --profile unknown.
- 	profileScript := filepath.Join(tempDir, "profile_cli.sh")
- 	if err := os.WriteFile(profileScript, []byte("#!/bin/sh\necho 'Error: unknown flag: --profile' >&2\nexit 1"), 0755); err != nil {
- 		t.Fatalf("failed to create profile script: %v", err)
- 		t.Fatalf("failed to create host script: %v", err)
- 	}
- 
--	ts := &CliTokenSource{
--		forceCmd:   []string{forceScript},
--		profileCmd: []string{profileScript},
--		hostCmd:    []string{hostScript},
--	}
-+	ts := newTestCliTokenSource([]cliCommand{
-+		{args: []string{forceScript}, usedFlags: []string{"--force-refresh", "--profile"}, warningMessage: "force-refresh not supported"},
-+		{args: []string{profileScript}, usedFlags: []string{"--profile"}, warningMessage: "profile not supported"},
-+		{args: []string{hostScript}},
-+	})
- 	token, err := ts.Token(context.Background())
- 	if err != nil {
- 		t.Fatalf("Token() error = %v, want fallback through to hostCmd to succeed", err)
- 
- 	tempDir := t.TempDir()
- 
--	// forceCmd fails with a real auth error (not unknown flag).
- 	forceScript := filepath.Join(tempDir, "force_cli.sh")
- 	if err := os.WriteFile(forceScript, []byte("#!/bin/sh\necho 'cache: databricks OAuth is not configured for this host' >&2\nexit 1"), 0755); err != nil {
- 		t.Fatalf("failed to create force script: %v", err)
- 	}
- 
--	// profileCmd and hostCmd should not be called.
-+	// Subsequent commands should not be called.
- 	profileScript := filepath.Join(tempDir, "profile_cli.sh")
- 	if err := os.WriteFile(profileScript, []byte("#!/bin/sh\necho 'should not reach here' >&2\nexit 1"), 0755); err != nil {
- 		t.Fatalf("failed to create profile script: %v", err)
- 		t.Fatalf("failed to create host script: %v", err)
- 	}
- 
--	ts := &CliTokenSource{
--		forceCmd:   []string{forceScript},
--		profileCmd: []string{profileScript},
--		hostCmd:    []string{hostScript},
--	}
-+	ts := newTestCliTokenSource([]cliCommand{
-+		{args: []string{forceScript}, usedFlags: []string{"--force-refresh", "--profile"}, warningMessage: "force-refresh not supported"},
-+		{args: []string{profileScript}, usedFlags: []string{"--profile"}, warningMessage: "profile not supported"},
-+		{args: []string{hostScript}},
-+	})
- 	_, err := ts.Token(context.Background())
- 	if err == nil {
- 		t.Fatal("Token() error = nil, want error")
++			ts := newTestCliTokenSource(commands)
+ 			token, err := ts.Token(context.Background())
+ 			if tc.wantErrMsg != "" {
+ 				if err == nil || !strings.Contains(err.Error(), tc.wantErrMsg) {
+ 		})
  	}
  }
- 
--func TestCliTokenSource_Token_NilHostCmdReturnsError(t *testing.T) {
++
 +func TestCliTokenSource_Token_ActiveCommandIndexPersists(t *testing.T) {
- 	if runtime.GOOS == "windows" {
- 		t.Skip("Skipping shell script test on Windows")
- 	}
- 
++	if runtime.GOOS == "windows" {
++		t.Skip("Skipping shell script test on Windows")
++	}
++
 +	expiry := time.Now().Add(1 * time.Hour).Format(time.RFC3339)
 +	validResponse, _ := json.Marshal(cliTokenResponse{
 +		AccessToken: "host-token",
@@ -247,38 +213,27 @@
 +		Expiry:      expiry,
 +	})
 +
- 	tempDir := t.TempDir()
- 
- 	forceScript := filepath.Join(tempDir, "force_cli.sh")
--	if err := os.WriteFile(forceScript, []byte("#!/bin/sh\necho 'Error: unknown flag: --profile' >&2\nexit 1"), 0755); err != nil {
++	tempDir := t.TempDir()
++
++	forceScript := filepath.Join(tempDir, "force_cli.sh")
 +	if err := os.WriteFile(forceScript, []byte("#!/bin/sh\necho 'Error: unknown flag: --force-refresh' >&2\nexit 1"), 0755); err != nil {
- 		t.Fatalf("failed to create force script: %v", err)
- 	}
- 
--	profileScript := filepath.Join(tempDir, "profile_cli.sh")
--	if err := os.WriteFile(profileScript, []byte("#!/bin/sh\necho 'Error: unknown flag: --profile' >&2\nexit 1"), 0755); err != nil {
--		t.Fatalf("failed to create profile script: %v", err)
++		t.Fatalf("failed to create force script: %v", err)
++	}
++
 +	hostScript := filepath.Join(tempDir, "host_cli.sh")
 +	if err := os.WriteFile(hostScript, []byte("#!/bin/sh\necho '"+string(validResponse)+"'"), 0755); err != nil {
 +		t.Fatalf("failed to create host script: %v", err)
- 	}
- 
--	ts := &CliTokenSource{
--		forceCmd:   []string{forceScript},
--		profileCmd: []string{profileScript},
++	}
++
 +	ts := newTestCliTokenSource([]cliCommand{
 +		{args: []string{forceScript}, usedFlags: []string{"--force-refresh"}, warningMessage: "force-refresh not supported"},
 +		{args: []string{hostScript}},
 +	})
 +
-+	// First call: falls back from forceScript to hostScript.
 +	token, err := ts.Token(context.Background())
 +	if err != nil {
 +		t.Fatalf("first Token() error = %v", err)
- 	}
--	_, err := ts.Token(context.Background())
--	if err == nil {
--		t.Fatal("Token() error = nil, want error")
++	}
 +	if token.AccessToken != "host-token" {
 +		t.Errorf("first AccessToken = %q, want %q", token.AccessToken, "host-token")
 +	}
@@ -286,14 +241,11 @@
 +		t.Errorf("activeCommandIndex = %d, want 1", ts.activeCommandIndex.Load())
 +	}
 +
-+	// Second call: starts at activeCommandIndex, skipping the force command.
 +	token, err = ts.Token(context.Background())
 +	if err != nil {
 +		t.Fatalf("second Token() error = %v", err)
- 	}
--	if !strings.Contains(err.Error(), "no CLI commands available") {
--		t.Errorf("Token() error = %v, want error containing %q", err, "no CLI commands available")
++	}
 +	if token.AccessToken != "host-token" {
 +		t.Errorf("second AccessToken = %q, want %q", token.AccessToken, "host-token")
- 	}
- }
\ No newline at end of file
++	}
++}
\ No newline at end of file

_{Reproduce locally: git range-diff 69a7c95..79ac794 a314000..a19dab0 | Disable: git config gitstack.push-range-diff false}

mihaimitrea-db · 2026-04-13T15:23:18Z

Update: Reopening this PR with a new approach.

The try-and-retry fallback mechanism from the previous iteration does not work for `--profile` because it is a global Cobra flag — old CLIs (< v0.207.1) silently accept it instead of reporting `"unknown flag: --profile"`, failing later with `"cannot fetch credentials"`. This made the `--host` fallback dead code.

The new approach uses version detection: run `databricks version` at init time, parse the semver, and use it to decide which flags the CLI supports. This also enables passing `--force-refresh` (databricks/cli#4767) when the CLI version supports it.

See the updated PR description for details.

The --profile flag is a global Cobra flag in the Databricks CLI. Old CLIs (< v0.207.1) silently accept it on `auth token` but fail with "cannot fetch credentials" instead of "unknown flag: --profile". This made the previous error-based fallback to --host dead code. Replace the try-and-retry approach with version-based detection: run `databricks version` at init time and use the parsed semver to decide between --profile and --host. This also simplifies CliTokenSource to a single resolved command with no runtime probing. Signed-off-by: Mihai Mitrea <mihai.mitrea@databricks.com>

github-actions · 2026-04-14T14:33:42Z

If integration tests don't run automatically, an authorized user can run them manually by following the instructions below:

Trigger:
go/deco-tests-run/sdk-go

Inputs:

PR number: 1605
Commit SHA: 271785e92cb795074b3d6e7af841889fd84497d5

Checks will be approved automatically on success.

mihaimitrea-db mentioned this pull request Mar 30, 2026

Add --force-refresh flag to CLI token source #1604

Closed

mihaimitrea-db temporarily deployed to test-trigger-is March 30, 2026 13:33 — with GitHub Actions Inactive

mihaimitrea-db temporarily deployed to test-trigger-is March 30, 2026 13:35 — with GitHub Actions Inactive

mihaimitrea-db force-pushed the mihaimitrea-db/stack/generalize-cli-commands branch from 4cb4c26 to 82d9599 Compare March 30, 2026 13:45

mihaimitrea-db temporarily deployed to test-trigger-is March 30, 2026 13:45 — with GitHub Actions Inactive

mihaimitrea-db force-pushed the mihaimitrea-db/stack/generalize-cli-commands branch from 82d9599 to 68d45f4 Compare March 30, 2026 13:56

mihaimitrea-db temporarily deployed to test-trigger-is March 30, 2026 13:56 — with GitHub Actions Inactive

mihaimitrea-db temporarily deployed to test-trigger-is March 30, 2026 13:57 — with GitHub Actions Inactive

mihaimitrea-db force-pushed the mihaimitrea-db/stack/generalize-cli-commands branch from 68d45f4 to 4a5079f Compare March 30, 2026 15:14

mihaimitrea-db temporarily deployed to test-trigger-is March 30, 2026 15:14 — with GitHub Actions Inactive

mihaimitrea-db had a problem deploying to test-trigger-is March 30, 2026 15:14 — with GitHub Actions Failure

mihaimitrea-db temporarily deployed to test-trigger-is March 30, 2026 15:34 — with GitHub Actions Inactive

mihaimitrea-db force-pushed the mihaimitrea-db/stack/generalize-cli-commands branch from 4a5079f to 6f4fead Compare March 30, 2026 16:27

mihaimitrea-db temporarily deployed to test-trigger-is March 30, 2026 16:27 — with GitHub Actions Inactive

mihaimitrea-db force-pushed the mihaimitrea-db/stack/generalize-cli-commands branch from 6f4fead to 2218270 Compare March 31, 2026 08:07

mihaimitrea-db temporarily deployed to test-trigger-is March 31, 2026 08:07 — with GitHub Actions Inactive

mihaimitrea-db temporarily deployed to test-trigger-is March 31, 2026 08:08 — with GitHub Actions Inactive

mihaimitrea-db force-pushed the mihaimitrea-db/stack/generalize-cli-commands branch from 2218270 to 76e74ca Compare March 31, 2026 09:06

mihaimitrea-db temporarily deployed to test-trigger-is March 31, 2026 09:06 — with GitHub Actions Inactive

mihaimitrea-db temporarily deployed to test-trigger-is March 31, 2026 09:07 — with GitHub Actions Inactive

mihaimitrea-db marked this pull request as ready for review March 31, 2026 09:53

mihaimitrea-db requested a review from renaudhartert-db March 31, 2026 09:53

mihaimitrea-db mentioned this pull request Apr 1, 2026

Generalize CLI token source into progressive command list databricks/databricks-sdk-py#1378

Draft

renaudhartert-db reviewed Apr 1, 2026

View reviewed changes

mihaimitrea-db force-pushed the mihaimitrea-db/stack/generalize-cli-commands branch from 79ac794 to a19dab0 Compare April 7, 2026 09:18

mihaimitrea-db temporarily deployed to test-trigger-is April 7, 2026 09:18 — with GitHub Actions Inactive

mihaimitrea-db had a problem deploying to test-trigger-is April 7, 2026 09:20 — with GitHub Actions Failure

mihaimitrea-db requested a review from renaudhartert-db April 8, 2026 14:00

mihaimitrea-db closed this Apr 13, 2026

This was referenced Apr 14, 2026

Fix CLI token source --profile fallback with version detection #1624

Closed

Pass --force-refresh to CLI auth token command #1625

Closed

mihaimitrea-db reopened this Apr 14, 2026

mihaimitrea-db force-pushed the mihaimitrea-db/stack/generalize-cli-commands branch from a19dab0 to 6c77eda Compare April 14, 2026 13:39

mihaimitrea-db temporarily deployed to test-trigger-is April 14, 2026 13:39 — with GitHub Actions Inactive

mihaimitrea-db changed the title ~~Generalize CLI token source into progressive command list~~ Fix CLI token source --profile fallback with version detection Apr 14, 2026

mihaimitrea-db had a problem deploying to test-trigger-is April 14, 2026 13:41 — with GitHub Actions Failure

mihaimitrea-db mentioned this pull request Apr 14, 2026

Pass --force-refresh to CLI auth token command #1628

Open

mihaimitrea-db temporarily deployed to test-trigger-is April 14, 2026 14:08 — with GitHub Actions Inactive

mihaimitrea-db had a problem deploying to test-trigger-is April 14, 2026 14:09 — with GitHub Actions Failure

mihaimitrea-db force-pushed the mihaimitrea-db/stack/generalize-cli-commands branch from d7431c4 to 0572f6e Compare April 14, 2026 14:22

mihaimitrea-db temporarily deployed to test-trigger-is April 14, 2026 14:22 — with GitHub Actions Inactive

mihaimitrea-db had a problem deploying to test-trigger-is April 14, 2026 14:22 — with GitHub Actions Failure

mihaimitrea-db force-pushed the mihaimitrea-db/stack/generalize-cli-commands branch from 0572f6e to 4280bbe Compare April 14, 2026 14:29

mihaimitrea-db temporarily deployed to test-trigger-is April 14, 2026 14:30 — with GitHub Actions Inactive

mihaimitrea-db had a problem deploying to test-trigger-is April 14, 2026 14:30 — with GitHub Actions Failure

mihaimitrea-db force-pushed the mihaimitrea-db/stack/generalize-cli-commands branch from 4280bbe to 271785e Compare April 14, 2026 14:33

mihaimitrea-db temporarily deployed to test-trigger-is April 14, 2026 14:33 — with GitHub Actions Inactive

mihaimitrea-db had a problem deploying to test-trigger-is April 14, 2026 14:34 — with GitHub Actions Failure

	// one of its [cliCommand.usedFlags], it logs a warning and tries the next.
	// one of its [cliCommand.usedFlags], it logs a warning and tries the next. The "counter" is not incremented if the command fails for any other reason.

Conversation

mihaimitrea-db commented Mar 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Why

Approaches considered

References

What changed

Interface changes

Behavioral changes

Internal changes

How is this tested?

Uh oh!

mihaimitrea-db commented Mar 30, 2026

Uh oh!

mihaimitrea-db commented Mar 30, 2026

Uh oh!

mihaimitrea-db commented Mar 30, 2026

Uh oh!

mihaimitrea-db commented Mar 30, 2026

Uh oh!

mihaimitrea-db commented Mar 31, 2026

Uh oh!

mihaimitrea-db commented Mar 31, 2026

Uh oh!

renaudhartert-db Mar 31, 2026

Choose a reason for hiding this comment

Uh oh!

This comment was marked as outdated.

Uh oh!

renaudhartert-db Apr 1, 2026

Choose a reason for hiding this comment

Uh oh!

renaudhartert-db Apr 1, 2026

Choose a reason for hiding this comment

Uh oh!

mihaimitrea-db commented Apr 7, 2026

Uh oh!

mihaimitrea-db commented Apr 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Apr 14, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

mihaimitrea-db commented Mar 30, 2026 •

edited

Loading

mihaimitrea-db commented Apr 13, 2026 •

edited

Loading